{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"NLU_training_multi_class_text_classifier_demo_musical_instruments.ipynb","provenance":[],"collapsed_sections":[]},"kernelspec":{"display_name":"Python 3","name":"python3"}},"cells":[{"cell_type":"markdown","metadata":{"id":"zkufh760uvF3"},"source":["![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)\n","\n","[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/nlu/blob/master/examples/colab/Training/multi_class_text_classification/NLU_training_multi_class_text_classifier_demo_musical_instruments.ipynb)\n","\n","\n","\n","\n","# Training a Deep Learning Classifier with NLU \n","## ClassifierDL (Multi-class Text Classification)\n","## 4 class Amazon Musical Instruments review classifier training\n","With the [ClassifierDL model](https://nlp.johnsnowlabs.com/docs/en/annotators#classifierdl-multi-class-text-classification) from Spark NLP you can achieve State Of the Art results on any multi class text classification problem \n","\n","This notebook showcases the following features : \n","\n","- How to train the deep learning classifier\n","- How to store a pipeline to disk\n","- How to load the pipeline from disk (Enables NLU offline mode)\n","\n","You can achieve these results or even better on this dataset with training data:\n","\n","<br>\n","\n","\n","![image.png]()\n","\n","You can achieve these results or even better on this dataset with test data:\n","\n","<br>\n","\n","![image.png]()\n"]},{"cell_type":"markdown","metadata":{"id":"dur2drhW5Rvi"},"source":["# 1. Install Java 8 and NLU"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"hFGnBCHavltY","executionInfo":{"status":"ok","timestamp":1620190616877,"user_tz":-300,"elapsed":36961,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"}},"outputId":"54af3f90-c6c3-45ce-cb9f-ba8c06124414"},"source":["!wget https://setup.johnsnowlabs.com/nlu/colab.sh -O - | bash\n","  \n","\n","import nlu"],"execution_count":null,"outputs":[{"output_type":"stream","text":["--2021-05-05 04:56:20--  https://raw.githubusercontent.com/JohnSnowLabs/nlu/master/scripts/colab_setup.sh\n","Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.108.133, ...\n","Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.\n","HTTP request sent, awaiting response... 200 OK\n","Length: 1671 (1.6K) [text/plain]\n","Saving to: ‘STDOUT’\n","\n","\r-                     0%[                    ]       0  --.-KB/s               Installing  NLU 3.0.0 with  PySpark 3.0.2 and Spark NLP 3.0.1 for Google Colab ...\n","\r-                   100%[===================>]   1.63K  --.-KB/s    in 0.001s  \n","\n","2021-05-05 04:56:20 (1.60 MB/s) - written to stdout [1671/1671]\n","\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"f4KkTfnR5Ugg"},"source":["# 2. Download musical instruments  classification dataset\n","\n","https://www.kaggle.com/eswarchandt/amazon-music-reviews\n","\n","dataset with products rated between 5 classes"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"OrVb5ZMvvrQD","executionInfo":{"status":"ok","timestamp":1620190617767,"user_tz":-300,"elapsed":35618,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"}},"outputId":"5bd3e397-592e-4790-983a-b5e9449132f6"},"source":["! wget http://ckl-it.de/wp-content/uploads/2021/01/Musical_instruments_reviews.csv"],"execution_count":null,"outputs":[{"output_type":"stream","text":["--2021-05-05 04:56:56--  http://ckl-it.de/wp-content/uploads/2021/01/Musical_instruments_reviews.csv\n","Resolving ckl-it.de (ckl-it.de)... 217.160.0.108, 2001:8d8:100f:f000::209\n","Connecting to ckl-it.de (ckl-it.de)|217.160.0.108|:80... connected.\n","HTTP request sent, awaiting response... 200 OK\n","Length: 51708 (50K) [text/csv]\n","Saving to: ‘Musical_instruments_reviews.csv.1’\n","\n","Musical_instruments 100%[===================>]  50.50K   220KB/s    in 0.2s    \n","\n","2021-05-05 04:56:57 (220 KB/s) - ‘Musical_instruments_reviews.csv.1’ saved [51708/51708]\n","\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":419},"id":"y4xSRWIhwT28","executionInfo":{"status":"ok","timestamp":1620190618447,"user_tz":-300,"elapsed":33313,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"}},"outputId":"ce3a288d-579a-4cd6-910a-7da1b55e85ac"},"source":["import pandas as pd\n","test_path = '/content/Musical_instruments_reviews.csv'\n","train_df = pd.read_csv(test_path,sep=\",\")\n","cols = [\"y\",\"text\"]\n","train_df = train_df[cols]\n","from sklearn.model_selection import train_test_split\n","train_df, test_df = train_test_split(train_df, test_size=0.2)\n","train_df\n","\n"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>y</th>\n","      <th>text</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>11</th>\n","      <td>very poor</td>\n","      <td>I know many people love the design, but I find...</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>good</td>\n","      <td>Well made. Works as it should. However, seem t...</td>\n","    </tr>\n","    <tr>\n","      <th>45</th>\n","      <td>very good</td>\n","      <td>The product does exactly as it should and is q...</td>\n","    </tr>\n","    <tr>\n","      <th>25</th>\n","      <td>average</td>\n","      <td>This is a fine guitar, but it isn't amazing.  ...</td>\n","    </tr>\n","    <tr>\n","      <th>97</th>\n","      <td>average</td>\n","      <td>The problem with this pedal is that you have t...</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>70</th>\n","      <td>very poor</td>\n","      <td>I might have done something wrong...I read in ...</td>\n","    </tr>\n","    <tr>\n","      <th>31</th>\n","      <td>good</td>\n","      <td>Good product at a good price. Have used it mul...</td>\n","    </tr>\n","    <tr>\n","      <th>48</th>\n","      <td>average</td>\n","      <td>Doe's not stay on to well, moves to much even ...</td>\n","    </tr>\n","    <tr>\n","      <th>78</th>\n","      <td>very poor</td>\n","      <td>Go build your own. Build it to your specs and ...</td>\n","    </tr>\n","    <tr>\n","      <th>39</th>\n","      <td>good</td>\n","      <td>I've been using these for about 3 weeks now - ...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>96 rows × 2 columns</p>\n","</div>"],"text/plain":["            y                                               text\n","11  very poor  I know many people love the design, but I find...\n","10       good  Well made. Works as it should. However, seem t...\n","45  very good  The product does exactly as it should and is q...\n","25    average  This is a fine guitar, but it isn't amazing.  ...\n","97    average  The problem with this pedal is that you have t...\n","..        ...                                                ...\n","70  very poor  I might have done something wrong...I read in ...\n","31       good  Good product at a good price. Have used it mul...\n","48    average  Doe's not stay on to well, moves to much even ...\n","78  very poor  Go build your own. Build it to your specs and ...\n","39       good  I've been using these for about 3 weeks now - ...\n","\n","[96 rows x 2 columns]"]},"metadata":{"tags":[]},"execution_count":3}]},{"cell_type":"markdown","metadata":{"id":"0296Om2C5anY"},"source":["# 3. Train Deep Learning Classifier using nlu.load('train.classifier')\n","\n","By default, the Universal Sentence Encoder Embeddings (USE) are beeing downloaded to provide embeddings for the classifier. You can use any of the 50+ other sentence Emeddings in NLU tough!\n","\n","You dataset label column should be named 'y' and the feature column with text data should be named 'text'"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"id":"3ZIPkRkWftBG","executionInfo":{"elapsed":278961,"status":"ok","timestamp":1620189909399,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"},"user_tz":-300},"outputId":"a72a6d7e-b133-43fc-a6ce-b49185a1cba5"},"source":["# load a trainable pipeline by specifying the train. prefix  and fit it on a datset with label and text columns\n","# Since there are no\n","\n","trainable_pipe = nlu.load('train.classifier')\n","fitted_pipe = trainable_pipe.fit(train_df.iloc[:50] )\n","\n","\n","# predict with the trainable pipeline on dataset and get predictions\n","preds = fitted_pipe.predict(train_df.iloc[:50],output_level='document' )\n","preds"],"execution_count":null,"outputs":[{"output_type":"stream","text":["tfhub_use download started this may take some time.\n","Approximate size to download 923.7 MB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>sentence</th>\n","      <th>trained_classifier</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>document</th>\n","      <th>text</th>\n","      <th>y</th>\n","      <th>sentence_embedding_use</th>\n","      <th>origin_index</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>[Very good cable., Well made and it looks grea...</td>\n","      <td>very poor</td>\n","      <td>0.344603</td>\n","      <td>Very good cable. Well made and it looks great ...</td>\n","      <td>Very good cable. Well made and it looks great ...</td>\n","      <td>good</td>\n","      <td>[0.018388595432043076, -0.001244456390850246, ...</td>\n","      <td>76</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>[I bought this hoping to use as intended, angl...</td>\n","      <td>very poor</td>\n","      <td>0.532565</td>\n","      <td>I bought this hoping to use as intended, angle...</td>\n","      <td>I bought this hoping to use as intended, angle...</td>\n","      <td>very poor</td>\n","      <td>[0.07512206584215164, 0.010702813044190407, -0...</td>\n","      <td>53</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>[Hosa cable quality can be all over the place,...</td>\n","      <td>good</td>\n","      <td>0.413519</td>\n","      <td>Hosa cable quality can be all over the place, ...</td>\n","      <td>Hosa cable quality can be all over the place, ...</td>\n","      <td>good</td>\n","      <td>[0.05411579832434654, 0.04756820946931839, -0....</td>\n","      <td>112</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>[wanted it just on looks alone., ., ., It is a...</td>\n","      <td>good</td>\n","      <td>0.350552</td>\n","      <td>wanted it just on looks alone...It is a nice l...</td>\n","      <td>wanted it just on looks alone...It is a nice l...</td>\n","      <td>very good</td>\n","      <td>[0.08269041031599045, 0.02812184765934944, -0....</td>\n","      <td>79</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>[Well made., Works as it should., However, see...</td>\n","      <td>good</td>\n","      <td>0.453055</td>\n","      <td>Well made. Works as it should. However, seem t...</td>\n","      <td>Well made. Works as it should. However, seem t...</td>\n","      <td>good</td>\n","      <td>[0.07170691341161728, -0.012493303045630455, -...</td>\n","      <td>10</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>[Nice solid cables, with excellent support at ...</td>\n","      <td>good</td>\n","      <td>0.435087</td>\n","      <td>Nice solid cables, with excellent support at t...</td>\n","      <td>Nice solid cables, with excellent support at t...</td>\n","      <td>very good</td>\n","      <td>[0.06005071476101875, 0.0279176477342844, -0.0...</td>\n","      <td>20</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>[If you are not use to using a large sustainin...</td>\n","      <td>average</td>\n","      <td>0.627601</td>\n","      <td>If you are not use to using a large sustaining...</td>\n","      <td>If you are not use to using a large sustaining...</td>\n","      <td>average</td>\n","      <td>[0.06641201674938202, 0.07415173947811127, -0....</td>\n","      <td>83</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>[The Hosa XLR cables are affordable and very h...</td>\n","      <td>good</td>\n","      <td>0.451387</td>\n","      <td>The Hosa XLR cables are affordable and very he...</td>\n","      <td>The Hosa XLR cables are affordable and very he...</td>\n","      <td>good</td>\n","      <td>[0.0650528222322464, -0.024019034579396248, -0...</td>\n","      <td>117</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>[Not much to write about here, but it does exa...</td>\n","      <td>good</td>\n","      <td>0.375702</td>\n","      <td>Not much to write about here, but it does exac...</td>\n","      <td>Not much to write about here, but it does exac...</td>\n","      <td>very good</td>\n","      <td>[0.06030378118157387, 0.03140004724264145, -0....</td>\n","      <td>89</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>[The primary job of this device is to block th...</td>\n","      <td>average</td>\n","      <td>0.445224</td>\n","      <td>The primary job of this device is to block the...</td>\n","      <td>The primary job of this device is to block the...</td>\n","      <td>very good</td>\n","      <td>[0.0768439993262291, 0.012965132482349873, -0....</td>\n","      <td>116</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>[Only Lasted 2yr., I have a lot of cheaper cab...</td>\n","      <td>good</td>\n","      <td>0.370069</td>\n","      <td>Only Lasted 2yr. I have a lot of cheaper cable...</td>\n","      <td>Only Lasted 2yr.  I have a lot of cheaper cabl...</td>\n","      <td>very poor</td>\n","      <td>[0.05348164588212967, 0.03492450341582298, -0....</td>\n","      <td>55</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>[These cables are a little thin compared to ho...</td>\n","      <td>average</td>\n","      <td>0.581801</td>\n","      <td>These cables are a little thin compared to hos...</td>\n","      <td>These cables are a little thin compared to hos...</td>\n","      <td>average</td>\n","      <td>[0.05915072187781334, -0.03095736913383007, -0...</td>\n","      <td>3</td>\n","    </tr>\n","    <tr>\n","      <th>12</th>\n","      <td>[This pedal has been around for a long time, a...</td>\n","      <td>average</td>\n","      <td>0.963964</td>\n","      <td>This pedal has been around for a long time, an...</td>\n","      <td>This pedal has been around for a long time, an...</td>\n","      <td>average</td>\n","      <td>[0.06582090258598328, 0.012904703617095947, -0...</td>\n","      <td>102</td>\n","    </tr>\n","    <tr>\n","      <th>13</th>\n","      <td>[Fender cords look great and work just as well...</td>\n","      <td>average</td>\n","      <td>0.874885</td>\n","      <td>Fender cords look great and work just as well....</td>\n","      <td>Fender cords look great and work just as well....</td>\n","      <td>very good</td>\n","      <td>[0.018997572362422943, -0.012294900603592396, ...</td>\n","      <td>95</td>\n","    </tr>\n","    <tr>\n","      <th>14</th>\n","      <td>[It just randomly pops off my bass, it's so sl...</td>\n","      <td>very poor</td>\n","      <td>0.731916</td>\n","      <td>It just randomly pops off my bass, it's so sli...</td>\n","      <td>It just randomly pops off my bass, it's so sli...</td>\n","      <td>very poor</td>\n","      <td>[0.018867170438170433, 0.05410968139767647, -0...</td>\n","      <td>115</td>\n","    </tr>\n","    <tr>\n","      <th>15</th>\n","      <td>[the string work well but did not give the mel...</td>\n","      <td>average</td>\n","      <td>0.712628</td>\n","      <td>the string work well but did not give the mell...</td>\n","      <td>the string work well but did not give the mell...</td>\n","      <td>average</td>\n","      <td>[0.056391388177871704, 0.07034655660390854, -0...</td>\n","      <td>36</td>\n","    </tr>\n","    <tr>\n","      <th>16</th>\n","      <td>[I've used a lot of cables and I always come b...</td>\n","      <td>good</td>\n","      <td>0.440745</td>\n","      <td>I've used a lot of cables and I always come ba...</td>\n","      <td>I've used a lot of cables and I always come ba...</td>\n","      <td>very good</td>\n","      <td>[0.07708822190761566, 0.016957569867372513, -0...</td>\n","      <td>108</td>\n","    </tr>\n","    <tr>\n","      <th>17</th>\n","      <td>[Long story short, this string set has a stron...</td>\n","      <td>average</td>\n","      <td>0.767563</td>\n","      <td>Long story short, this string set has a strong...</td>\n","      <td>Long story short, this string set has a strong...</td>\n","      <td>very poor</td>\n","      <td>[0.06738117337226868, 0.05691230669617653, -0....</td>\n","      <td>88</td>\n","    </tr>\n","    <tr>\n","      <th>18</th>\n","      <td>[Monster makes a wide array of cables, includi...</td>\n","      <td>average</td>\n","      <td>0.465245</td>\n","      <td>Monster makes a wide array of cables, includin...</td>\n","      <td>Monster makes a wide array of cables, includin...</td>\n","      <td>very good</td>\n","      <td>[0.06651592999696732, 0.0612233430147171, -0.0...</td>\n","      <td>59</td>\n","    </tr>\n","    <tr>\n","      <th>19</th>\n","      <td>[...unbalanced guitar cable is notoriously noi...</td>\n","      <td>average</td>\n","      <td>0.511957</td>\n","      <td>...unbalanced guitar cable is notoriously nois...</td>\n","      <td>...unbalanced guitar cable is notoriously nois...</td>\n","      <td>average</td>\n","      <td>[0.07284880429506302, 0.024346861988306046, 0....</td>\n","      <td>52</td>\n","    </tr>\n","    <tr>\n","      <th>20</th>\n","      <td>[Nice windscreen protects my MXL mic and preve...</td>\n","      <td>average</td>\n","      <td>0.410393</td>\n","      <td>Nice windscreen protects my MXL mic and preven...</td>\n","      <td>Nice windscreen protects my MXL mic and preven...</td>\n","      <td>very good</td>\n","      <td>[0.07302771508693695, -0.04292221739888191, -0...</td>\n","      <td>14</td>\n","    </tr>\n","    <tr>\n","      <th>21</th>\n","      <td>[I bought 6 of these XLR to 1/4&amp;#34; adapters ...</td>\n","      <td>good</td>\n","      <td>0.436735</td>\n","      <td>I bought 6 of these XLR to 1/4&amp;#34; adapters h...</td>\n","      <td>I bought 6 of these XLR to 1/4&amp;#34; adapters h...</td>\n","      <td>good</td>\n","      <td>[0.049845140427351, -0.02525583654642105, -0.0...</td>\n","      <td>16</td>\n","    </tr>\n","    <tr>\n","      <th>22</th>\n","      <td>[This Fender cable is the perfect length for m...</td>\n","      <td>good</td>\n","      <td>0.417587</td>\n","      <td>This Fender cable is the perfect length for me...</td>\n","      <td>This Fender cable is the perfect length for me...</td>\n","      <td>good</td>\n","      <td>[0.06648766994476318, 0.04505337029695511, -0....</td>\n","      <td>57</td>\n","    </tr>\n","    <tr>\n","      <th>23</th>\n","      <td>[This L Jack is good enough to make connection...</td>\n","      <td>average</td>\n","      <td>0.330708</td>\n","      <td>This L Jack is good enough to make connections...</td>\n","      <td>This L Jack is good enough to make connections...</td>\n","      <td>good</td>\n","      <td>[0.08174190670251846, 0.07509452849626541, -0....</td>\n","      <td>46</td>\n","    </tr>\n","    <tr>\n","      <th>24</th>\n","      <td>[These came stock on my Jackson Kelly KEXMG., ...</td>\n","      <td>average</td>\n","      <td>0.437556</td>\n","      <td>These came stock on my Jackson Kelly KEXMG. Al...</td>\n","      <td>These came stock on my Jackson Kelly KEXMG. Al...</td>\n","      <td>very poor</td>\n","      <td>[0.03714499622583389, 0.0533314123749733, -0.0...</td>\n","      <td>18</td>\n","    </tr>\n","    <tr>\n","      <th>25</th>\n","      <td>[Good overall but pick ups are terrible - Migh...</td>\n","      <td>average</td>\n","      <td>0.922614</td>\n","      <td>Good overall but pick ups are terrible - Might...</td>\n","      <td>Good overall but pick ups are terrible - Might...</td>\n","      <td>average</td>\n","      <td>[0.07406474649906158, 0.007126895245164633, -0...</td>\n","      <td>56</td>\n","    </tr>\n","    <tr>\n","      <th>26</th>\n","      <td>[These things are terrible., One wouldn't fit ...</td>\n","      <td>good</td>\n","      <td>0.348053</td>\n","      <td>These things are terrible. One wouldn't fit in...</td>\n","      <td>These things are terrible. One wouldn't fit in...</td>\n","      <td>very poor</td>\n","      <td>[0.05325164273381233, -0.036399971693754196, -...</td>\n","      <td>101</td>\n","    </tr>\n","    <tr>\n","      <th>27</th>\n","      <td>[another item never received but i would not u...</td>\n","      <td>very poor</td>\n","      <td>0.638240</td>\n","      <td>another item never received but i would not us...</td>\n","      <td>another item never received but i would not us...</td>\n","      <td>very poor</td>\n","      <td>[0.024511804804205894, 0.055792637169361115, 0...</td>\n","      <td>26</td>\n","    </tr>\n","    <tr>\n","      <th>28</th>\n","      <td>[wind screen is way too big its bulky and to m...</td>\n","      <td>very poor</td>\n","      <td>0.438548</td>\n","      <td>wind screen is way too big its bulky and to me...</td>\n","      <td>wind screen is way too big its bulky and to me...</td>\n","      <td>very poor</td>\n","      <td>[0.07763350009918213, 0.04169292375445366, -0....</td>\n","      <td>63</td>\n","    </tr>\n","    <tr>\n","      <th>29</th>\n","      <td>[This is a fine guitar, but it isn't amazing.,...</td>\n","      <td>average</td>\n","      <td>0.988021</td>\n","      <td>This is a fine guitar, but it isn't amazing. M...</td>\n","      <td>This is a fine guitar, but it isn't amazing.  ...</td>\n","      <td>average</td>\n","      <td>[0.05681991949677467, 0.055675093084573746, -0...</td>\n","      <td>25</td>\n","    </tr>\n","    <tr>\n","      <th>30</th>\n","      <td>[I've been using these for about 3 weeks now -...</td>\n","      <td>average</td>\n","      <td>0.411459</td>\n","      <td>I've been using these for about 3 weeks now - ...</td>\n","      <td>I've been using these for about 3 weeks now - ...</td>\n","      <td>good</td>\n","      <td>[0.07338722050189972, 0.024855980649590492, 2....</td>\n","      <td>39</td>\n","    </tr>\n","    <tr>\n","      <th>31</th>\n","      <td>[It's a good OD, but I thought it would be hig...</td>\n","      <td>average</td>\n","      <td>0.996302</td>\n","      <td>It's a good OD, but I thought it would be high...</td>\n","      <td>It's a good OD, but I thought it would be high...</td>\n","      <td>average</td>\n","      <td>[0.08331827074289322, -0.011826762929558754, -...</td>\n","      <td>80</td>\n","    </tr>\n","    <tr>\n","      <th>32</th>\n","      <td>[This is a cheap piece of junk that does what ...</td>\n","      <td>average</td>\n","      <td>0.900313</td>\n","      <td>This is a cheap piece of junk that does what i...</td>\n","      <td>This is a cheap piece of junk that does what i...</td>\n","      <td>very poor</td>\n","      <td>[0.07090918719768524, -0.004987373482435942, 0...</td>\n","      <td>75</td>\n","    </tr>\n","    <tr>\n","      <th>33</th>\n","      <td>[Works for practice ., .. it's a guitar instru...</td>\n","      <td>average</td>\n","      <td>0.785570</td>\n","      <td>Works for practice ... it's a guitar instrumen...</td>\n","      <td>Works for practice ... it's a guitar instrumen...</td>\n","      <td>average</td>\n","      <td>[0.08485165238380432, 0.0717940405011177, -0.0...</td>\n","      <td>93</td>\n","    </tr>\n","    <tr>\n","      <th>34</th>\n","      <td>[Let me start by saying that I am a huge fan o...</td>\n","      <td>average</td>\n","      <td>0.950551</td>\n","      <td>Let me start by saying that I am a huge fan of...</td>\n","      <td>Let me start by saying that I am a huge fan of...</td>\n","      <td>average</td>\n","      <td>[0.06803068518638611, 0.03993965685367584, -0....</td>\n","      <td>62</td>\n","    </tr>\n","    <tr>\n","      <th>35</th>\n","      <td>[Cant go wrong.,  Great quality on a budget pr...</td>\n","      <td>very poor</td>\n","      <td>0.331621</td>\n","      <td>Cant go wrong. Great quality on a budget price...</td>\n","      <td>Cant go wrong. Great quality on a budget price...</td>\n","      <td>good</td>\n","      <td>[0.07220637053251266, 0.020649369806051254, -0...</td>\n","      <td>33</td>\n","    </tr>\n","    <tr>\n","      <th>36</th>\n","      <td>[this piece of plastic is just terrible., you'...</td>\n","      <td>average</td>\n","      <td>0.443282</td>\n","      <td>this piece of plastic is just terrible. you'd ...</td>\n","      <td>this piece of plastic is just terrible.  you'd...</td>\n","      <td>very poor</td>\n","      <td>[0.07407913357019424, 0.07586841285228729, 0.0...</td>\n","      <td>104</td>\n","    </tr>\n","    <tr>\n","      <th>37</th>\n","      <td>[It is a decent cable., It does its job, but i...</td>\n","      <td>average</td>\n","      <td>0.682894</td>\n","      <td>It is a decent cable. It does its job, but it ...</td>\n","      <td>It is a decent cable. It does its job, but it ...</td>\n","      <td>average</td>\n","      <td>[0.07582439482212067, 0.04041396453976631, -0....</td>\n","      <td>4</td>\n","    </tr>\n","    <tr>\n","      <th>38</th>\n","      <td>[For the price, fantastic., They do feel light...</td>\n","      <td>average</td>\n","      <td>0.433290</td>\n","      <td>For the price, fantastic.They do feel light an...</td>\n","      <td>For the price, fantastic.They do feel light an...</td>\n","      <td>good</td>\n","      <td>[0.07524136453866959, 0.049453943967819214, -0...</td>\n","      <td>99</td>\n","    </tr>\n","    <tr>\n","      <th>39</th>\n","      <td>[Just assembly for a try, only few rolls with ...</td>\n","      <td>very poor</td>\n","      <td>0.529660</td>\n","      <td>Just assembly for a try, only few rolls with m...</td>\n","      <td>Just assembly for a try, only few rolls with m...</td>\n","      <td>very poor</td>\n","      <td>[0.08081397414207458, 0.014145106077194214, -0...</td>\n","      <td>98</td>\n","    </tr>\n","    <tr>\n","      <th>40</th>\n","      <td>[I know many people love the design, but I fin...</td>\n","      <td>very poor</td>\n","      <td>0.639893</td>\n","      <td>I know many people love the design, but I find...</td>\n","      <td>I know many people love the design, but I find...</td>\n","      <td>very poor</td>\n","      <td>[0.06185115873813629, 0.06986682116985321, -0....</td>\n","      <td>11</td>\n","    </tr>\n","    <tr>\n","      <th>41</th>\n","      <td>[This cable seems like it will last me for a w...</td>\n","      <td>good</td>\n","      <td>0.449773</td>\n","      <td>This cable seems like it will last me for a wh...</td>\n","      <td>This cable seems like it will last me for a wh...</td>\n","      <td>good</td>\n","      <td>[0.08796018362045288, 0.015969790518283844, -0...</td>\n","      <td>42</td>\n","    </tr>\n","    <tr>\n","      <th>42</th>\n","      <td>[So good that I bought another one., Love the ...</td>\n","      <td>good</td>\n","      <td>0.378669</td>\n","      <td>So good that I bought another one. Love the he...</td>\n","      <td>So good that I bought another one.  Love the h...</td>\n","      <td>very good</td>\n","      <td>[0.05002278462052345, -0.013001631945371628, -...</td>\n","      <td>74</td>\n","    </tr>\n","    <tr>\n","      <th>43</th>\n","      <td>[Good quality cable and sounds very good]</td>\n","      <td>good</td>\n","      <td>0.418967</td>\n","      <td>Good quality cable and sounds very good</td>\n","      <td>Good quality cable and sounds very good</td>\n","      <td>very good</td>\n","      <td>[0.06367859989404678, -0.0017134330701082945, ...</td>\n","      <td>113</td>\n","    </tr>\n","    <tr>\n","      <th>44</th>\n","      <td>[Back in the 1980's I bought a can of deoxit(n...</td>\n","      <td>average</td>\n","      <td>0.983325</td>\n","      <td>Back in the 1980's I bought a can of deoxit(no...</td>\n","      <td>Back in the 1980's I bought a can of deoxit(no...</td>\n","      <td>average</td>\n","      <td>[0.06321641802787781, 0.04852892830967903, 0.0...</td>\n","      <td>34</td>\n","    </tr>\n","    <tr>\n","      <th>45</th>\n","      <td>[I bought this cord after returning a cheap on...</td>\n","      <td>good</td>\n","      <td>0.438915</td>\n","      <td>I bought this cord after returning a cheap one...</td>\n","      <td>I bought this cord after returning a cheap one...</td>\n","      <td>good</td>\n","      <td>[0.06954371929168701, 0.016654161736369133, -0...</td>\n","      <td>44</td>\n","    </tr>\n","    <tr>\n","      <th>46</th>\n","      <td>[It's a cable, no frills, tangles pretty easy ...</td>\n","      <td>average</td>\n","      <td>0.506431</td>\n","      <td>It's a cable, no frills, tangles pretty easy a...</td>\n","      <td>It's a cable, no frills, tangles pretty easy a...</td>\n","      <td>average</td>\n","      <td>[0.07703661173582077, 0.024149607867002487, -0...</td>\n","      <td>118</td>\n","    </tr>\n","    <tr>\n","      <th>47</th>\n","      <td>[I acknowledge that this is a minority opinion...</td>\n","      <td>average</td>\n","      <td>0.410722</td>\n","      <td>I acknowledge that this is a minority opinion ...</td>\n","      <td>I acknowledge that this is a minority opinion ...</td>\n","      <td>average</td>\n","      <td>[0.06933587044477463, 0.03066118247807026, -0....</td>\n","      <td>92</td>\n","    </tr>\n","    <tr>\n","      <th>48</th>\n","      <td>[I got one of these a while back and being lik...</td>\n","      <td>average</td>\n","      <td>0.972998</td>\n","      <td>I got one of these a while back and being like...</td>\n","      <td>I got one of these a while back and being like...</td>\n","      <td>average</td>\n","      <td>[0.06218786910176277, 0.019702719524502754, -0...</td>\n","      <td>85</td>\n","    </tr>\n","    <tr>\n","      <th>49</th>\n","      <td>[If you're like me, you probably bought this t...</td>\n","      <td>good</td>\n","      <td>0.444040</td>\n","      <td>If you're like me, you probably bought this to...</td>\n","      <td>If you're like me, you probably bought this to...</td>\n","      <td>good</td>\n","      <td>[0.08035349100828171, -0.013505871407687664, -...</td>\n","      <td>82</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                             sentence  ... origin_index\n","0   [Very good cable., Well made and it looks grea...  ...           76\n","1   [I bought this hoping to use as intended, angl...  ...           53\n","2   [Hosa cable quality can be all over the place,...  ...          112\n","3   [wanted it just on looks alone., ., ., It is a...  ...           79\n","4   [Well made., Works as it should., However, see...  ...           10\n","5   [Nice solid cables, with excellent support at ...  ...           20\n","6   [If you are not use to using a large sustainin...  ...           83\n","7   [The Hosa XLR cables are affordable and very h...  ...          117\n","8   [Not much to write about here, but it does exa...  ...           89\n","9   [The primary job of this device is to block th...  ...          116\n","10  [Only Lasted 2yr., I have a lot of cheaper cab...  ...           55\n","11  [These cables are a little thin compared to ho...  ...            3\n","12  [This pedal has been around for a long time, a...  ...          102\n","13  [Fender cords look great and work just as well...  ...           95\n","14  [It just randomly pops off my bass, it's so sl...  ...          115\n","15  [the string work well but did not give the mel...  ...           36\n","16  [I've used a lot of cables and I always come b...  ...          108\n","17  [Long story short, this string set has a stron...  ...           88\n","18  [Monster makes a wide array of cables, includi...  ...           59\n","19  [...unbalanced guitar cable is notoriously noi...  ...           52\n","20  [Nice windscreen protects my MXL mic and preve...  ...           14\n","21  [I bought 6 of these XLR to 1/4&#34; adapters ...  ...           16\n","22  [This Fender cable is the perfect length for m...  ...           57\n","23  [This L Jack is good enough to make connection...  ...           46\n","24  [These came stock on my Jackson Kelly KEXMG., ...  ...           18\n","25  [Good overall but pick ups are terrible - Migh...  ...           56\n","26  [These things are terrible., One wouldn't fit ...  ...          101\n","27  [another item never received but i would not u...  ...           26\n","28  [wind screen is way too big its bulky and to m...  ...           63\n","29  [This is a fine guitar, but it isn't amazing.,...  ...           25\n","30  [I've been using these for about 3 weeks now -...  ...           39\n","31  [It's a good OD, but I thought it would be hig...  ...           80\n","32  [This is a cheap piece of junk that does what ...  ...           75\n","33  [Works for practice ., .. it's a guitar instru...  ...           93\n","34  [Let me start by saying that I am a huge fan o...  ...           62\n","35  [Cant go wrong.,  Great quality on a budget pr...  ...           33\n","36  [this piece of plastic is just terrible., you'...  ...          104\n","37  [It is a decent cable., It does its job, but i...  ...            4\n","38  [For the price, fantastic., They do feel light...  ...           99\n","39  [Just assembly for a try, only few rolls with ...  ...           98\n","40  [I know many people love the design, but I fin...  ...           11\n","41  [This cable seems like it will last me for a w...  ...           42\n","42  [So good that I bought another one., Love the ...  ...           74\n","43          [Good quality cable and sounds very good]  ...          113\n","44  [Back in the 1980's I bought a can of deoxit(n...  ...           34\n","45  [I bought this cord after returning a cheap on...  ...           44\n","46  [It's a cable, no frills, tangles pretty easy ...  ...          118\n","47  [I acknowledge that this is a minority opinion...  ...           92\n","48  [I got one of these a while back and being lik...  ...           85\n","49  [If you're like me, you probably bought this t...  ...           82\n","\n","[50 rows x 8 columns]"]},"metadata":{"tags":[]},"execution_count":4}]},{"cell_type":"markdown","metadata":{"id":"DL_5aY9b3jSd"},"source":["# 4. Evaluate the model"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"djtoZVKBw2WU","executionInfo":{"elapsed":278954,"status":"ok","timestamp":1620189909402,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"},"user_tz":-300},"outputId":"c284dc4c-a2d8-4b4e-b493-e474f22175ff"},"source":["from sklearn.metrics import classification_report\n","print(classification_report(preds['y'], preds['classifier_dl']))"],"execution_count":null,"outputs":[{"output_type":"stream","text":["              precision    recall  f1-score   support\n","\n","     average       0.58      1.00      0.73        15\n","        good       0.50      0.62      0.55        13\n","   very good       0.00      0.00      0.00        10\n","   very poor       0.75      0.50      0.60        12\n","\n","    accuracy                           0.58        50\n","   macro avg       0.46      0.53      0.47        50\n","weighted avg       0.48      0.58      0.51        50\n","\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"mhFKVN93o1ZO"},"source":["# 5. Lets try different Sentence Emebddings"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"CzJd8omao0gt","executionInfo":{"elapsed":278946,"status":"ok","timestamp":1620189909403,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"},"user_tz":-300},"outputId":"691c5ce0-ce36-4082-9de9-070508bcbe9e"},"source":["# We can use nlu.print_components(action='embed_sentence') to see every possibler sentence embedding we could use. Lets use bert!\n","nlu.print_components(action='embed_sentence')"],"execution_count":null,"outputs":[{"output_type":"stream","text":["For language <en> NLU provides the following Models : \n","nlu.load('en.embed_sentence') returns Spark NLP model tfhub_use\n","nlu.load('en.embed_sentence.use') returns Spark NLP model tfhub_use\n","nlu.load('en.embed_sentence.tfhub_use') returns Spark NLP model tfhub_use\n","nlu.load('en.embed_sentence.use.lg') returns Spark NLP model tfhub_use_lg\n","nlu.load('en.embed_sentence.tfhub_use.lg') returns Spark NLP model tfhub_use_lg\n","nlu.load('en.embed_sentence.albert') returns Spark NLP model albert_base_uncased\n","nlu.load('en.embed_sentence.electra') returns Spark NLP model sent_electra_small_uncased\n","nlu.load('en.embed_sentence.electra_small_uncased') returns Spark NLP model sent_electra_small_uncased\n","nlu.load('en.embed_sentence.electra_base_uncased') returns Spark NLP model sent_electra_base_uncased\n","nlu.load('en.embed_sentence.electra_large_uncased') returns Spark NLP model sent_electra_large_uncased\n","nlu.load('en.embed_sentence.bert') returns Spark NLP model sent_bert_base_uncased\n","nlu.load('en.embed_sentence.bert_base_uncased') returns Spark NLP model sent_bert_base_uncased\n","nlu.load('en.embed_sentence.bert_base_cased') returns Spark NLP model sent_bert_base_cased\n","nlu.load('en.embed_sentence.bert_large_uncased') returns Spark NLP model sent_bert_large_uncased\n","nlu.load('en.embed_sentence.bert_large_cased') returns Spark NLP model sent_bert_large_cased\n","nlu.load('en.embed_sentence.biobert.pubmed_base_cased') returns Spark NLP model sent_biobert_pubmed_base_cased\n","nlu.load('en.embed_sentence.biobert.pubmed_large_cased') returns Spark NLP model sent_biobert_pubmed_large_cased\n","nlu.load('en.embed_sentence.biobert.pmc_base_cased') returns Spark NLP model sent_biobert_pmc_base_cased\n","nlu.load('en.embed_sentence.biobert.pubmed_pmc_base_cased') returns Spark NLP model sent_biobert_pubmed_pmc_base_cased\n","nlu.load('en.embed_sentence.biobert.clinical_base_cased') returns Spark NLP model sent_biobert_clinical_base_cased\n","nlu.load('en.embed_sentence.biobert.discharge_base_cased') returns Spark NLP model sent_biobert_discharge_base_cased\n","nlu.load('en.embed_sentence.covidbert.large_uncased') returns Spark NLP model sent_covidbert_large_uncased\n","nlu.load('en.embed_sentence.small_bert_L2_128') returns Spark NLP model sent_small_bert_L2_128\n","nlu.load('en.embed_sentence.small_bert_L4_128') returns Spark NLP model sent_small_bert_L4_128\n","nlu.load('en.embed_sentence.small_bert_L6_128') returns Spark NLP model sent_small_bert_L6_128\n","nlu.load('en.embed_sentence.small_bert_L8_128') returns Spark NLP model sent_small_bert_L8_128\n","nlu.load('en.embed_sentence.small_bert_L10_128') returns Spark NLP model sent_small_bert_L10_128\n","nlu.load('en.embed_sentence.small_bert_L12_128') returns Spark NLP model sent_small_bert_L12_128\n","nlu.load('en.embed_sentence.small_bert_L2_256') returns Spark NLP model sent_small_bert_L2_256\n","nlu.load('en.embed_sentence.small_bert_L4_256') returns Spark NLP model sent_small_bert_L4_256\n","nlu.load('en.embed_sentence.small_bert_L6_256') returns Spark NLP model sent_small_bert_L6_256\n","nlu.load('en.embed_sentence.small_bert_L8_256') returns Spark NLP model sent_small_bert_L8_256\n","nlu.load('en.embed_sentence.small_bert_L10_256') returns Spark NLP model sent_small_bert_L10_256\n","nlu.load('en.embed_sentence.small_bert_L12_256') returns Spark NLP model sent_small_bert_L12_256\n","nlu.load('en.embed_sentence.small_bert_L2_512') returns Spark NLP model sent_small_bert_L2_512\n","nlu.load('en.embed_sentence.small_bert_L4_512') returns Spark NLP model sent_small_bert_L4_512\n","nlu.load('en.embed_sentence.small_bert_L6_512') returns Spark NLP model sent_small_bert_L6_512\n","nlu.load('en.embed_sentence.small_bert_L8_512') returns Spark NLP model sent_small_bert_L8_512\n","nlu.load('en.embed_sentence.small_bert_L10_512') returns Spark NLP model sent_small_bert_L10_512\n","nlu.load('en.embed_sentence.small_bert_L12_512') returns Spark NLP model sent_small_bert_L12_512\n","nlu.load('en.embed_sentence.small_bert_L2_768') returns Spark NLP model sent_small_bert_L2_768\n","nlu.load('en.embed_sentence.small_bert_L4_768') returns Spark NLP model sent_small_bert_L4_768\n","nlu.load('en.embed_sentence.small_bert_L6_768') returns Spark NLP model sent_small_bert_L6_768\n","nlu.load('en.embed_sentence.small_bert_L8_768') returns Spark NLP model sent_small_bert_L8_768\n","nlu.load('en.embed_sentence.small_bert_L10_768') returns Spark NLP model sent_small_bert_L10_768\n","nlu.load('en.embed_sentence.small_bert_L12_768') returns Spark NLP model sent_small_bert_L12_768\n","For language <fi> NLU provides the following Models : \n","nlu.load('fi.embed_sentence') returns Spark NLP model sent_bert_finnish_cased\n","nlu.load('fi.embed_sentence.bert.cased') returns Spark NLP model sent_bert_finnish_cased\n","nlu.load('fi.embed_sentence.bert.uncased') returns Spark NLP model sent_bert_finnish_uncased\n","For language <xx> NLU provides the following Models : \n","nlu.load('xx.embed_sentence') returns Spark NLP model sent_bert_multi_cased\n","nlu.load('xx.embed_sentence.bert') returns Spark NLP model sent_bert_multi_cased\n","nlu.load('xx.embed_sentence.bert.cased') returns Spark NLP model sent_bert_multi_cased\n","nlu.load('xx.embed_sentence.labse') returns Spark NLP model labse\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"ABHLgirmG1n9","executionInfo":{"elapsed":483071,"status":"ok","timestamp":1620190113536,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"},"user_tz":-300},"outputId":"2c9fbc78-da3b-4c61-f6b2-bb876789cd7a"},"source":["# Load pipe with bert embeds\n","# using large embeddings can take a few hours..\n","# fitted_pipe = nlu.load('en.embed_sentence.bert_large_uncased train.classifier').fit(train_df)\n","fitted_pipe = nlu.load('en.embed_sentence.bert train.classifier').fit(train_df.iloc[:100])\n","\n","\n","# predict with the trained pipeline on dataset and get predictions\n","preds = fitted_pipe.predict(train_df.iloc[:100],output_level='document')\n","from sklearn.metrics import classification_report\n","print(classification_report(preds['y'], preds['classifier_dl']))\n"],"execution_count":null,"outputs":[{"output_type":"stream","text":["sent_bert_base_uncased download started this may take some time.\n","Approximate size to download 392.5 MB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n","              precision    recall  f1-score   support\n","\n","     average       0.00      0.00      0.00        23\n","        good       0.00      0.00      0.00        25\n","   very good       0.00      0.00      0.00        22\n","   very poor       0.27      1.00      0.43        26\n","\n","    accuracy                           0.27        96\n","   macro avg       0.07      0.25      0.11        96\n","weighted avg       0.07      0.27      0.12        96\n","\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"nbpdZGoZPslz","executionInfo":{"elapsed":507540,"status":"ok","timestamp":1620190138012,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"},"user_tz":-300},"outputId":"1680223b-0587-44b2-9d83-a1b87736129b"},"source":["# Load pipe with bert embeds\n","fitted_pipe = nlu.load('embed_sentence.bert train.classifier').fit(train_df.iloc[:100])\n","\n","# predict with the trained pipeline on dataset and get predictions\n","preds = fitted_pipe.predict(train_df.iloc[:100],output_level='document')\n","from sklearn.metrics import classification_report\n","print(classification_report(preds['y'], preds['classifier_dl']))\n"],"execution_count":null,"outputs":[{"output_type":"stream","text":["sent_small_bert_L2_128 download started this may take some time.\n","Approximate size to download 16.1 MB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n","              precision    recall  f1-score   support\n","\n","     average       0.24      1.00      0.39        23\n","        good       0.00      0.00      0.00        25\n","   very good       0.00      0.00      0.00        22\n","   very poor       0.00      0.00      0.00        26\n","\n","    accuracy                           0.24        96\n","   macro avg       0.06      0.25      0.10        96\n","weighted avg       0.06      0.24      0.09        96\n","\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"wYV7ivdsQY8Z","executionInfo":{"status":"ok","timestamp":1620190883307,"user_tz":-300,"elapsed":291703,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"}},"outputId":"8ad7cc16-cfe2-4ebd-b16f-16755062ca0e"},"source":["from sklearn.metrics import classification_report\n","trainable_pipe = nlu.load('en.embed_sentence.small_bert_L12_768 train.classifier')\n","# We need to train longer and user smaller LR for NON-USE based sentence embeddings usually\n","# We could tune the hyperparameters further with hyperparameter tuning methods like gridsearch\n","# Also longer training gives more accuracy\n","trainable_pipe['trainable_classifier_dl'].setMaxEpochs(90)  \n","trainable_pipe['trainable_classifier_dl'].setLr(0.0005) \n","fitted_pipe = trainable_pipe.fit(train_df)\n","# predict with the trainable pipeline on dataset and get predictions\n","preds = fitted_pipe.predict(train_df,output_level='document')\n","\n","#sentence detector that is part of the pipe generates sone NaNs. lets drop them first\n","preds.dropna(inplace=True)\n","print(classification_report(preds['y'], preds['classifier_dl']))\n","\n","#preds"],"execution_count":null,"outputs":[{"output_type":"stream","text":["sent_small_bert_L12_768 download started this may take some time.\n","Approximate size to download 392.9 MB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n","              precision    recall  f1-score   support\n","\n","     average       0.26      1.00      0.41        25\n","        good       0.00      0.00      0.00        24\n","   very good       0.00      0.00      0.00        23\n","   very poor       0.00      0.00      0.00        24\n","\n","    accuracy                           0.26        96\n","   macro avg       0.07      0.25      0.10        96\n","weighted avg       0.07      0.26      0.11        96\n","\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"_1jxw3GnVGlI"},"source":["#  6. evaluate on Test Data"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"Fxx4yNkNVGFl","executionInfo":{"status":"ok","timestamp":1620190892875,"user_tz":-300,"elapsed":300897,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"}},"outputId":"5dbda6ed-5884-4d26-ed5b-9f8d774558da"},"source":["preds = fitted_pipe.predict(test_df,output_level='document')\n","\n","#sentence detector that is part of the pipe generates sone NaNs. lets drop them first\n","preds.dropna(inplace=True)\n","print(classification_report(preds['y'], preds['classifier_dl']))"],"execution_count":null,"outputs":[{"output_type":"stream","text":["              precision    recall  f1-score   support\n","\n","     average       0.22      1.00      0.36         5\n","        good       0.00      0.00      0.00         6\n","   very good       0.00      0.00      0.00         7\n","   very poor       1.00      0.17      0.29         6\n","\n","    accuracy                           0.25        24\n","   macro avg       0.30      0.29      0.16        24\n","weighted avg       0.30      0.25      0.15        24\n","\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"2BB-NwZUoHSe"},"source":["# 7. Lets save the model"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"eLex095goHwm","executionInfo":{"status":"ok","timestamp":1620191279090,"user_tz":-300,"elapsed":262699,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"}},"outputId":"71745bf2-b5be-41e2-e1b4-d3683e113b57"},"source":["stored_model_path = './models/classifier_dl_trained' \n","fitted_pipe.save(stored_model_path)"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Stored model in ./model/classifier_dl_trained\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"e_b2DPd4rCiU"},"source":["# 8. Lets load the model from HDD.\n","This makes Offlien NLU usage possible!   \n","You need to call nlu.load(path=path_to_the_pipe) to load a model/pipeline from disk."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":80},"id":"SO4uz45MoRgp","executionInfo":{"status":"ok","timestamp":1620191295930,"user_tz":-300,"elapsed":206694,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"}},"outputId":"84cd50b8-77b5-44e2-ca60-efc36a592891"},"source":["hdd_pipe = nlu.load(path=stored_model_path)\n","\n","preds = hdd_pipe.predict('It was really good ')\n","preds"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>text</th>\n","      <th>origin_index</th>\n","      <th>sentence_embedding_from_disk</th>\n","      <th>document</th>\n","      <th>from_disk_confidence_confidence</th>\n","      <th>from_disk</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>It was really good</td>\n","      <td>8589934592</td>\n","      <td>[[-0.03466350957751274, 0.33072206377983093, 0...</td>\n","      <td>It was really good</td>\n","      <td>[0.53862494]</td>\n","      <td>[very poor]</td>\n","      <td>[It was really good]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                  text  origin_index  ...    from_disk              sentence\n","0  It was really good     8589934592  ...  [very poor]  [It was really good]\n","\n","[1 rows x 7 columns]"]},"metadata":{"tags":[]},"execution_count":8}]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"e0CVlkk9v6Qi","executionInfo":{"status":"ok","timestamp":1620191295931,"user_tz":-300,"elapsed":206555,"user":{"displayName":"Gammer Otaku","photoUrl":"","userId":"18042713576744284398"}},"outputId":"c92becb6-6751-47de-a500-d2d8a371d545"},"source":["hdd_pipe.print_info()"],"execution_count":null,"outputs":[{"output_type":"stream","text":["The following parameters are configurable for this NLU pipeline (You can copy paste the examples) :\n",">>> pipe['document_assembler'] has settable params:\n","pipe['document_assembler'].setCleanupMode('shrink')                                                      | Info: possible values: disabled, inplace, inplace_full, shrink, shrink_full, each, each_full, delete_full | Currently set to : shrink\n",">>> pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'] has settable params:\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setExplodeSentences(False)                | Info: whether to explode each sentence into a different row, for better parallelization. Defaults to false. | Currently set to : False\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setStorageRef('SentenceDetectorDLModel_c83c27f46b97')  | Info: storage unique identifier | Currently set to : SentenceDetectorDLModel_c83c27f46b97\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setEncoder(com.johnsnowlabs.nlp.annotators.sentence_detector_dl.SentenceDetectorDLEncoder@6ee9116a)  | Info: Data encoder | Currently set to : com.johnsnowlabs.nlp.annotators.sentence_detector_dl.SentenceDetectorDLEncoder@6ee9116a\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setImpossiblePenultimates(['Bros', 'No', 'al', 'vs', 'etc', 'Fig', 'Dr', 'Prof', 'PhD', 'MD', 'Co', 'Corp', 'Inc', 'bros', 'VS', 'Vs', 'ETC', 'fig', 'dr', 'prof', 'PHD', 'phd', 'md', 'co', 'corp', 'inc', 'Jan', 'Feb', 'Mar', 'Apr', 'Jul', 'Aug', 'Sep', 'Sept', 'Oct', 'Nov', 'Dec', 'St', 'st', 'AM', 'PM', 'am', 'pm', 'e.g', 'f.e', 'i.e'])  | Info: Impossible penultimates | Currently set to : ['Bros', 'No', 'al', 'vs', 'etc', 'Fig', 'Dr', 'Prof', 'PhD', 'MD', 'Co', 'Corp', 'Inc', 'bros', 'VS', 'Vs', 'ETC', 'fig', 'dr', 'prof', 'PHD', 'phd', 'md', 'co', 'corp', 'inc', 'Jan', 'Feb', 'Mar', 'Apr', 'Jul', 'Aug', 'Sep', 'Sept', 'Oct', 'Nov', 'Dec', 'St', 'st', 'AM', 'PM', 'am', 'pm', 'e.g', 'f.e', 'i.e']\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setModelArchitecture('cnn')               | Info: Model architecture (CNN) | Currently set to : cnn\n",">>> pipe['bert_sentence@sent_small_bert_L12_768'] has settable params:\n","pipe['bert_sentence@sent_small_bert_L12_768'].setBatchSize(8)                                            | Info: Size of every batch | Currently set to : 8\n","pipe['bert_sentence@sent_small_bert_L12_768'].setCaseSensitive(False)                                    | Info: whether to ignore case in tokens for embeddings matching | Currently set to : False\n","pipe['bert_sentence@sent_small_bert_L12_768'].setDimension(768)                                          | Info: Number of embedding dimensions | Currently set to : 768\n","pipe['bert_sentence@sent_small_bert_L12_768'].setMaxSentenceLength(128)                                  | Info: Max sentence length to process | Currently set to : 128\n","pipe['bert_sentence@sent_small_bert_L12_768'].setIsLong(False)                                           | Info: Use Long type instead of Int type for inputs buffer - Some Bert models require Long instead of Int. | Currently set to : False\n","pipe['bert_sentence@sent_small_bert_L12_768'].setStorageRef('sent_small_bert_L12_768')                   | Info: unique reference name for identification | Currently set to : sent_small_bert_L12_768\n",">>> pipe['classifier_dl@sent_small_bert_L12_768'] has settable params:\n","pipe['classifier_dl@sent_small_bert_L12_768'].setClasses(['very good', 'very poor', 'average', 'good'])  | Info: get the tags used to trained this ClassifierDLModel | Currently set to : ['very good', 'very poor', 'average', 'good']\n","pipe['classifier_dl@sent_small_bert_L12_768'].setStorageRef('sent_small_bert_L12_768')                   | Info: unique reference name for identification | Currently set to : sent_small_bert_L12_768\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"iJBrKFd94n8y"},"source":[""],"execution_count":null,"outputs":[]}]}